The Shape of Failure

Part of the difficulty in constructing highly available services is that a modern Internet service is mind-boggling in complexity. The application, database, operating system, and firmware contain thousands of interfaces and millions of lines of code. Worse, unlike many other highly available systems, the major components are created by different organizations with potentially different goals and reliability parameters; no single entity has authority over the design and implementation of the entire system. Thus, ensuring traditional fault-tolerant techniques are followed top-to-bottom is a difficult, if not impossible, organizational task.